Model-driven development of data intensive applications over cloud resources

The proliferation of sensors over the last years has generated large amounts of raw data, forming data streams that need to be processed. In many cases, cloud resources are used for such processing, exploiting their flexibility, but these sensor streaming applications often need to support operational and control actions that have real-time and low-latency requirements that go beyond the cost effective and flexible solutions supported by existing cloud frameworks, such as Apache Kafka, Apache Spark Streaming, or Map-Reduce Streams. In this paper, we describe a model-driven and stepwise refinement methodological approach for streaming applications executed over clouds. The central role is assigned to a set of Petri Net models for specifying functional and non-functional requirements. They support model reuse, and a way to combine formal analysis, simulation, and approximate computation of minimal and maximal boundaries of non-functional requirements when the problem is either mathematically or computationally intractable. We show how our proposal can assist developers in their design and implementation decisions from a performance perspective. Our methodology allows to conduct performance analysis: The methodology is intended for all the engineering process stages, and we can (i) analyse how it can be mapped onto cloud resources, and (ii) obtain key performance indicators, including throughput or economic cost, so that developers are assisted in their development tasks and in their decision taking. In order to illustrate our approach, we make use of the pipelined wavefront array

Automatización del despliegue de simulaciones distribuidas en cloud híbrido

Computer Science for Complex System Modeling (COSMOS) es un grupo de investigación dentro del Departamento de Informática e Ingeniería de Sistemas (DIIS) que trabaja en el desarrollo de sistemas distribuidos. Uno de los proyectos que se encuentra bajo desarrollo es la simulación de Sistema de Eventos Discretos (SED) con una gran amplitud. Su foco principal en estos últimos an ̃os es la puesta en marcha de un simulador de SED en un entorno distribuido. En este TFG se propone una prueba para el desarrollo y automatización del despliegue de una infraestructura para la ejecución de simulaciones distribuidas en un entorno de Cloud Híbrido.Uno de los problemas que se afronta cuando este equipo se encuentra ante simulaciones con tamaño amplio es la incapacidad de realizar estas mismas en un único nodo, este problema se solventa con el uso de un simulador distribuido. La solución anterior genera de nuevo un obstáculo que consiste en que la operativa para la puesta en marcha de este simulador y la posterior recogida de datos es complicada y poco cómoda, por lo que es aquí donde entra en juego el Trabajo de Fin de Grado (TFG) que se documenta en esta memoria.El proyecto que se desarrolla proporciona una solución para facilitar la operativa de las simulaciones. Esto se consigue con el uso de Slurm, un planificador de tareas desplegado en Cloud Híbrido. Elegir un planificador de software libre no ha sido una decisión fácil, puesto que, hoy en d ́ıa, existen múltiples herramientas en el mercado, todas ellas similares y muy completas. Tras seleccionar el planificador m ́as adecuado, se ha podido desplegar el sistema y elaborar una solución. La solución propone una infraestructura heterogénea que hace uso de Google Cloud Platform (GCP) como proveedor del cluster en Cloud Público, y, una cluster on-premise como proveedor del Cloud Privado.El despliegue del sistema en este entorno h ́ıbrido se encuentra automatizado para facilitar su futura puesta en marcha en otras infraestructuras. Esta automatización ha sido realizada haciendo uso de Terraform y Ansible, dos programas de modelización de recursos. El primero ha sido usado para los recursos Cloud Público (máquinas virtuales, zonas DNS, firewall, etc), mientras que el segundo ha sido usado para los recursos de administración de sistemas del Cloud Privado.Para terminar se han realizado dos tipos de evaluaciones. En primer lugar, se han validado los módulos de automatización de creación y modelización de recursos, y de esta manera se ha demostrado que los módulos desarrollados llevan a cabo su trabajo de manera correcta. Y, en segundo lugar, se han llevado a cabo pruebas con distintos casos de uso de despliegues de simulaciones en Cloud Privado, Cloud Público y Cloud Híbrido para comprobar la robustez y las capacidades del sistema desplegado. Una vez validado el sistema se han hecho pruebas en un entorno real de simulación distribuida. Tras comprobar que las simulaciones se ejecutaban de manera correcta con una asignación de recursos adecuada, y se podía obtener la salida de estas simulaciones de manera centralizada y sencilla, se dio por evaluado de manera correcta el sistema.Como resultado de ese proyecto, no solo se ha elaborado un TFG, sino que también se ha colaborado en la publicación de un articulo de investigación que ha sido aprobado, publicado y seleccionado como mejor artículo de la conferencia en la GECON - International Conference on the Economics of Grids, Clouds, Systems, and Services. Este proyecto servir ́a como base para futuros usuarios, profesores y grupos de investigación que deseen seguir la linea de investigación del despliegue de una infraestructura para simulaciones distribuidas en un entorno de Cloud Híbrido.<br /

A Specification Language for Performance and Economical Analysis of Short Term Data Intensive Energy Management Services

Requirements of Energy Management Services include short and long term processing of data in a massively interconnected scenario. The complexity and variety of short term applications needs methodologies that allow designers to reason about the models taking into account functional and non-functional requirements. In this paper we present a component based specification language for building trustworthy continuous dataflow applications. Component behaviour is defined by Petri Nets in order to translate to the methodology all the advantages derived from a mathematically based executable model to support analysis, verification, simulation and performance evaluation. The paper illustrates how to model and reason with specifications of advanced dataflow abstractions such as smart grids

Desarrollo de un Framework de simulación de sistemas de eventos discretos complejos

El diseño de sistemas complejos de grandes dimensiones requiere la definición de modelos de diferentes perspectivas y con diferentes niveles de abstracción. En este proyecto se han usado las redes de Petri como modelo formal. Una metodología modular y jerárquica de construcción de modelos es la más adecuada para gestionar la complejidad donde la definición de componentes y la comunicación/sincronización entre ellos juega un papel central. El objetivo es la generación de código eficiente para simulación distribuida con soporte al balanceo de carga, e implementar algoritmos de sincronización específicos en simulación que aseguren su corrección.<br /

Enforcing QoS in scientific workflow systems enacted over Cloud infrastructures

AbstractThe ability to support Quality of Service (QoS) constraints is an important requirement in some scientific applications. With the increasing use of Cloud computing infrastructures, where access to resources is shared, dynamic and provisioned on-demand, identifying how QoS constraints can be supported becomes an important challenge. However, access to dedicated resources is often not possible in existing Cloud deployments and limited QoS guarantees are provided by many commercial providers (often restricted to error rate and availability, rather than particular QoS metrics such as latency or access time). We propose a workflow system architecture which enforces QoS for the simultaneous execution of multiple scientific workflows over a shared infrastructure (such as a Cloud environment). Our approach involves multiple pipeline workflow instances, with each instance having its own QoS requirements. These workflows are composed of a number of stages, with each stage being mapped to one or more physical resources. A stage involves a combination of data access, computation and data transfer capability. A token bucket-based data throttling framework is embedded into the workflow system architecture. Each workflow instance stage regulates the amount of data that is injected into the shared resources, allowing for bursts of data to be injected while at the same time providing isolation of workflow streams. We demonstrate our approach by using the Montage workflow, and develop a Reference net model of the workflow

Characterising resource management performance in Kubernetes

A key challenge for supporting elastic behaviour in cloud systems is to achieve a good performance in automated (de-)provisioning and scheduling of computing resources. One of the key aspects that can be significant is the overheads associated with deploying, terminating and maintaining resources. Therefore, due to their lower start up and termination overhead, containers are rapidly replacing Virtual Machines (VMs) in many cloud deployments, as the computation instance of choice. In this paper, we analyse the performance of Kubernetes achieved through a Petri net-based performance model. Kubernetes is a container management system for a distributed cluster environment. Our model can be characterised using data from a Kubernetes deployment, and can be exploited for supporting capacity planning and designing Kubernetes-based elastic applications

Posidonia oceanica (L.) Delile in its westernmost biogeographical limit (northwestern Alboran Sea): Meadows characterisation, phenology and flowering events

Posidonia oceanica is a Mediterranean endemic seagrass species that forms meadows covering ca. 2.5–4.5 millions of hectares, representing ca.25 % of the infralittoral and shallow circalittoral (down to 50m) bottoms of the Mediterranean. This seagrass is considered a habitat-engineer species and provides an elevated number of ecosystem services. In addition the Marine Strategy Framework Directive (MSFD, 2008/56/EC) includes seagrass like elements to evaluate the “Good Environmental Status” of the European coasts. Information about their phenological characteristic and structure of the meadows is needed for indicator estimations in order to establish their conservation status. The studied meadows are located in the westernmost limit of the P. oceanica distribution (North-western Alboran Sea) in the vecinity of the Strait of Gibraltar, an Atlantic-Mediterranean water transition area. Four sites were selected from East to West: Paraje Natural de Acantilados de Maro-Cerro Gordo (hereafter Maro), Special Area of Conservation “Calahonda” (hereafter Calahonda), Site of Community Importance Estepona (hereafter Estepona) and Punta Chullera (hereafter Chullera) where P. oceanica present their westernmost meadows. Phenological data were recorded from mid November to mid December in P. oceanica patches located at 2 – 3 m depth. At each site three types of patches (patch area 2 m2, large patches) were sampled. At each patch and site, 3 quadrants of 45 x 45 cm were sampled for shoot and inflorescences density measurements. In each quadrant, 10 random shoots were sampled for shoot morphology (shoot height and number of leaves). Shoot and inflorescences densities were standardized to squared meters. All the studied P. oceanica meadows develop on rocks and they present a fragmented structure with a coverage ranging between ca. 45% in Calahonda and Estepona and ca. 31% in Maro. The meadows of Chullera are reduced to a few small - medium patches with areas ranging between 0.5-1.5 m2 (Fig. 1). The meadows of Chullera and Estepona presented similar values of shoot density (ca. 752 – 662 shoots m-2, respectively) and leaf height (ca. 25 cm). Similarly, the Calahonda and Maro meadows also showed similar values of shoot density (ca. 510 – 550 shoots m-2, respectively) but displaying lower values than those of sites located closer to the Strait of Gibraltar. Regarding patch sizes and leaf height, the longest leaves (ca. 25 cm) were found in medium and large patches, but the number of leaves per shoot were higher in the small and the medium size patches (ca. 6.3 leaves per shoot). Flowering was only detected at the Calahonda meadows with maximum values of ca. 330 inflorescences m-2 (115.2 ± 98.2 inflorescences m-2, n= 9; mean ± SD) (Fig.1). Inflorescence density was not significant different among patches of different sizes. In the Alboran Sea and unlike the studied meadows, extensive beds of P. oceanica occur at the National Park of Cabo de Gata (northeastern Alboran Sea), but from east to west (Strait of Gibraltar), meadows are gradually fragmenting and their depth range decrease from 30m to 2m depth between Cabo de Gata and Chullera, respectively. Probably, the Atlantic influence and the characteristic oceanographic conditions of the Alboran Sea (i.e., higher turbidity, higher water turbulence) represent a developmental limiting factor for P. oceanica at higher depths. Similarities between the meadows located closer to Strait of Gibraltar (Chullera and Estepona) were detected as well as between those more distant (Calahonda and Maro). The first ones showed higher values of shoot densities and leaf heights than the formers, which could be relating to the higher hydrodynamic exposure found at Chullera and Estepona meadows. Regarding flowering events, sexual reproduction in P. oceanica is not common in different locations of the Mediterranean Sea. The available information seems to indicate that flowering represent an irregular event and it is related to high seawater temperature. In fact, the flowering episodes that occurred in Calahonda in November 2015, match with the warmest year ever recorded. This is the third flowering event registered in these meadows located close to the westernmost distributional limit of P. oceanica (Málaga, Alboran Sea), which could indicates that these meadows presents a healthy status. Furthermore, the absence of significant differences in relation to inflorescence density between patches of different sizes may be indicating that the fragmentation does not necessarily influence on the flowering of this seagrass species

Photoactive nano-confined Pt in titania nanotubes (Pt-TiNT) via microwave-assisted flow synthesis

Pt-TiNT with PtO nanoparticles dispersed within the lumen and interlayer spaces of titania nanotubes (TiNT) were prepared by a new process involving titanate nanosheets (TiNS) synthesis in an optimized microwave-assisted flow reactor, followed by ion-exchange with a Pt precursor, before triggering the titanate layer rolling to trap the Pt precursor clusters inside the titania nanotubes, followed by a thermal treatment. TEM, XRD, and Raman analyses confirm the total conversion of TiO2 into TiNS in 15 min at 120 °C and 4 bar, and the TiNS transformation into 181 nm-long TiNT with 10 and 6 nm outer and inner diameter, respectively. The 2% Pt-TiNT comprises 0.7 nm PtO clusters (according to XPS), causing slight distortions of the interlayer spaces, while a few larger 2–3 nm Pt clusters reside within the lumen. As a result, Pt-TiNT is 14-fold more active than TiNT for visible light (400–780 nm) photocatalytic oxidation of diclofenac under 2136 μW·cm−2 irradiation, and>1000-fold better than the uncatalyzed photoconversion reaction under 100 mW·cm−2 artificial solar lighting. In addition, nano-confinement of PtO clusters narrowed the bandgap of the TiNT, which, combined with its excellent absorptivity to harvest light, allowed a broader spectral range of photon energies to activate the photocatalyst.11 página